CLEARINGHOUSE  FOR  FEDERAL  SCIENTIFIC  AND  TECHNICAL  INFORMATION  CFSTI 

DOCUMENT  MANAGEMENT  R RANCH  410.11 


LIMITATIONS  IN  REPRODUCTION  QUALITY 


ACCESSION  0  (*04  LfOSU 


RT  I .  WE  REQRET  THAT  LEGIRI LITY  OF  THIS  DOCUMENT  IS  IN  PART 
**  UNSATISFACTORY.  REPRODUCTION  HAS  BEEN  MADE  FROM  BEST 
AVAILABLE  COPY. 

(3  2.  A  PORTION  OF  THE  ORIGINAL  DOCUMENT  CONTAINS  FINE  DETAIL 
M  WHICH  MAY  MAKE  READING  OF  PHOTOCOPY  DIFFICULT. 

□  S.  THE  ORIGINAL  DOCUMENT  CONTAINS  COLOR.  BUT  DISTRIBUTION 

COPIES  ARE  AVAILABLE  IN  BLACK-AND-WHITE  REPRODUCTION 
ONLY. 

□  4.  THE  INITIAL  DISTRIBUTION  COPIES  CONTAIN  COLOR  WHICH  WILL 

BE  SHOWN  IN  BLACK-AND-WHITE  WHEN  IT  IS  NECESSARY  TO 
REPRINT. 


S.  LIMITED  SUPPLY  ON  HANC  WHEN  EXHAUSTED,  DOCUMENT  WILL 
BE  AVAILABLE  IN  MICROFICHE  ONLY. 


I.  LIMITED  SUPPLY  ON  HAND:  WHEN  EXHAUSTED  DOCUMENT  WILL 
NOT  BE  AVAILABLE. 


Q  7.  DOCUMENT  IS  AVAILABLE  IN  MICROFICHE  ONLY. 

Q  I.  DOCUMENT  AVAILABLE  ON  LOAN  FROM  CFSTI  (  TT  DOCUMENTS  ONLY). 


NBS  1/14 


PROCESSOR: 


(Ml 


ON  THE  OPTIMAL  INVENTORY  EQUATION 

Richard  Bellman 
Irving  Ollcksberg 
Oliver  Qroaa 


P-572 


28  September  1954 


Approved  for  GTS 


pin.  d  jppam  fSM 

HARD  COPY 

$.  /-*-v 

MICROFICHE 

%o.u- 

flJ  rUmf  n  rJ 


AUG  2  7  1964 


DOCIRA  0 


<“H  I  I  IS 

tTOO  MAIN  $1.  .  SANTA  MONICA  •  CAIITOINIA 


Summary 


y  ij>  pe.c^ir^rro 

In. this  napwr  wn  firmnwr#  a  summary  y\o f  some  result? 
recently  obtained  in  connection  with  the  oroblem  of  optimal 
inventory.  Solutions  are  given  fdr  the  case  of  proportional 
costs  and  arbitrary  demand,  and  fbr  the  ca3e  where  the  costs 
are  linear  with  constant  terms  representing  administrative 
cost s. ^ 


ON  THE  OPTIMAL  INVENTORY  EQUATION 


Richard  Bellman- 
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51  •  Introduction 

The  purpose  of  this  note  la  to  present  some  results  con¬ 
cerning  the  functional  equations 
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which  are  respectively  one-  and  two-dimensional  versions  of 
equations  arising  from  the  "optimal  Inventory"  problem,  an 
Interesting  and  Important  economic  and  Industrial  problem.  We 
will  state  one  version  of  this  below. 


The  mathematical  formulation  of  the  problem  follows  that 
of  Arrow,  Harris  and  Marschak,L‘]  •  A  more  detailed  study,  con¬ 
taining  existence  and  uniqueness  theorems  and  a  discussion  of 
related  statistical  questions,  is  contained  in  Dvoretzky,  Kiefer 
and  Wolfowitz,  c«,  cfo  also  Bellman,  »]  • 

Up  to  the  present  time,  however,  no  general  analytic  solut¬ 
ion  of  (1)  has  been  given,  and  it  would  seem  that  a  solution  for 
arbitrary  functions  must  possess  a  quite  complicated  and  intricate 
structureo  Here  we  shall  present  a  solution  for  the  case  where 
functions,  g(y)  and  ply)  are  linear  functions  of  y,  but  where 
the  distribution  function  dG  is  arbotrary. 

Tlie  method  we  use  is  equally  applicable  to  a  number  of  other 
cases  as  well,  as  we  shall  indicate  briefly  below.  A  detailed  ac¬ 
count  of  these  results  is  in  the  course  of  preparation. 

« 

1 2 .  The  "Optimal  Inventory"  Problem. 

One  version  of  the  optimal  inventory  problem  is  the  follow¬ 
ing.  At  any  time  we  have  N  resources  in  quantities  x^,x^,,..,x^ 

We  may,  if  we  wish,  order  additional  quantities  of  these  resources 
bq  that  we  have  on  hand  quantities  y  ,y^ » •  •  ®  i  at  a  cost  of 
i  &(y< ) •  Having  had  this  opportunity  to  increase  our  supplies, 

*■(  *»  X  1 

we  face  an  unknown  demand  (s1  ,8^ , . . . , s^ ) ,  where  s^  is  the  demand 
for  the  i-th  resource,  with  a  known  distribution  function  dG 
This  demand  is  met  as  far  as  possible.  If,  however,  si>  y^,  we 
must  order  a  quantity  at  higher  prices,  to  meet  this  demand . 

Assuming  a  constant  discount  factor  a  in  the  cost  of  ordering 
at  each  subsequent  stage,  how  do  we  order  at  each  stage  so  as  to 
minimize  the  total  expected  cost  over  time? 


2 


In  the  one-dimensional  case  where  x  is  the  quantity  of 
the  one  resource  on  hand  at  any  particular  time,  and  f(x)  is 
total  expected  cost  using  an  optimal  ordering  policy,  it  is 
readily  verified  that  f(x)  satisfies  (1.1).  Similarly,  flx^.x^) 
the  corresponding  function  in  the  two-resource  case,  satisfies 

naU 

As  problems  in  the  theory  of  dynamic  programming,  see, 

CO  ,  .  r.  d  w  ,  the  important  aspect  of  the  solution  is  not  so 
much  the  cost  function  f(x)  as  the  structure  of  the  policy 
function  y(x),  the  amount  ordered  at  each  stages  Using  tech¬ 
niques  employed  in  the  treatment  of  functional  equations  of 
similar  type,  see  CO,  0*3  ,  we  obtain  the  solutions  to  the  above 
equations  under  the  assumptions  mentioned  above  and  discuss 
below  solutions  obtained  under  some  modified  assurant ions. 

§  3.  The  Solution  of  the  N-dim  r.s tonal  case . 

In  presenting  the  solution  of  the  N-Jinens  ional  case, 
we  shall  state  the  result  in  two-dimensional  terms  for  ease  of 
not  itior. ,  and  shuIl  make  some  inessential  simplifications  which 
permit  a  simple  statement  of  the  results. 

Theorem.  Let  us  as.-  u:  ,e  that 

(a)  £*(>’)  *  ^y,  Vi  ly)  “  dherc  ^  and  d,  are  positive 

cor.,  tur.ts . 

(b)  -  >(s1  .s^ Jds^s^,  $  >  0. 

(c)  SoS  st^(e1 ta2)ds1^s^  <  »  ,  i  -  1,2. 

(d)  u  <  a  <  1. 

Then  the  so  1  rtior.  to  (1  ,2 )  h,  as  follows.  Let  xi ,  i  •  1,2,  be 
defined  us  th e  unique  positive  solutions  of 
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provided  that  they  exist*  Then 

(a)  For  CK .x^<  3^,  we  choose  ■  x^. 
lb)  For  Xjj>  1?^,  we  choose  y^^  *  3c^» 

If  x^  does  not  exist .  the  a bove  prescription  is  valid  with 


Observe  that  the  form  of  the  solution  is  independent  of  the 
dimension,  and  has,  in  addition,  the  important  property  of  sub¬ 
optimality  0 

f  i*»  Idea  of  the  Proof  for  the  One-Dimensional  Case , 

Let  us  indicate  briefly  the  essence  of  the  proof,  using 
the  one-dimensional  case  as  our  model.  The  key  to  the  mathematical 
analysis  is  the  observation  that  in  the  equation 

u(x)  -  Min  v(x,y ) ,  (4.1) 

y>x 

whenever  the  minimum  occurs  at  a  finite  value  y>x ,  we  have,  for 
this  value  of  y  the  simultaneous  equations 

Vy(x,y )  ■  0,  u'(x)  -  vx(x,y)  (4.2) 

Utlizingthe  observation,  we  see  that,  under  the  linearity 
assumptions  of  the  above  theorem,  whenever  y>x  we  have  simultane¬ 
ously 
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(a)  k,-  a p,  )  (fls)  .is  ♦ 

1  i  ^ 

(b)  f(x)  -  -k,. 

Upon  t heie  two  equations,  the  remainder  of  the  nroof 
hinges o 

$5.  Further  Results,, 

The  same  techniques,  reinforced  by  strenuous  application  of 
the  method  of  successive  approximations,  are  applicable  to  more 
general  processes,  involving  nonlinear  cost  and  penalty  functions, 
time-lags,  and  so  forth „ 

A  case  of  particular  importance  is  that  where  the  cost  func¬ 
tion  has  the  form  ky  ♦  R,  where  y  is  the  amount  ordered  and  R  is 
a  fixed  "red  tape"  cost.  If  the  penalty  cost  retains  its  proportional 

form,  then  regardless  of  the  distribution  of  demand,  the  optimal 

♦  V 

policy  is  an  sS-policy  of  the  tyre  discussed  by  Arrow,  Harris  and 
Marschak ,  L'l This  is  also  true  if  we  allow  the  penalty  cost  to 
contain  a  similar  red-tape  term,  for  a  large  d ass  of  distribution 
functions  of  practical  importance. 

These  results,  together  with  proofs  and  extensions  of  the 
above  results  will  appear  in  a  detailed  exposition  to  appear  else¬ 
where  0 
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